Method and system for filtering shadow maps with sub-frame accumulation

ABSTRACT

A method of rendering an image of an environment is disclosed. Environment data for the environment is accessed. The environment data corresponds to a frame of a video. A plurality of subframes associated with the frame is determined. An angle for each of the plurality of subframes is determined. One or more lights corresponding to the environment are selected. For each light of the one or more lights, a shadow map is generated. The shadow map corresponds to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe. The image of the environment is rendered. The rendering includes using the generated shadow map for each light of the one or more lights.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.63/027,330, filed May 19, 2020, entitled “METHOD AND SYSTEM FORFILTERING SHADOW MAPS WITH SUB-FRAME ACCUMULATION,” which isincorporated by reference herein in its entirety.

TECHNICAL FIELD

The subject matter disclosed herein generally relates to the technicalfield of computer graphics systems, and in one specific example, tocomputer systems and methods for rendering shadows in computer graphics.

BACKGROUND OF THE INVENTION

Various tools exist to allow shadows to be added to computer graphics(shadow mapping). However, shadow mapping even at high resolution canintroduce noise in graphics. This may happen through the introduction ofspatial aliasing artifacts (e.g., jaggines) in a rendered graphic imagesince the coverage of a shadow map texel is often larger than a pixel.These artifacts can be smoothed out through filtering but at the cost oflosing precision, sharpness and introducing light leak. The artifactsare particularly visible on clean surfaces (e.g., a surface with onlylow frequency detail), and at grazing angles creating long stripes ofshadow. Due to the way the human visual system works, the artifacts maybe very noticeable during an animation, introducing noise.

BRIEF DESCRIPTION OF THE DRAWINGS

Features and advantages of example embodiments of the present disclosurewill become apparent from the following detailed description, taken incombination with the appended drawings, in which:

FIG. 1 is a schematic illustrating a shadow map rotation system, inaccordance with one embodiment;

FIG. 2A is a flowchart illustrating a rotated shadow map method using ashadow map rotation system, in accordance with one embodiment;

FIG. 2B is a flowchart illustrating a method for rendering an image fora subframe using a shadow map rotation system, in accordance with oneembodiment;

FIG. 3 is a schematic illustrating a visual representation of a seriesof frames and associated subframes of digital content within a shadowmap rotation system, in accordance with one embodiment;

FIG. 4A is a schematic illustrating a shadow map grid orientedhorizontally for a shadow map rotation system, in accordance with oneembodiment;

FIG. 4B is a schematic illustrating a shadow map grid oriented at anangle from horizontal for a shadow map rotation system, in accordancewith one embodiment;

FIG. 5 is a schematic illustration of a cube map shadow map at twodifferent rotations within a shadow map rotation system, in accordancewith an embodiment;

FIG. 6A is a schematic illustration of a rendered image associated witha subframe showing rough shadow edges within a shadow map rotationsystem, in accordance with an embodiment;

FIG. 6B is a schematic illustration of a rendered image associated witha subframe showing smooth shadow edges within a shadow map rotationsystem, in accordance with an embodiment;

FIG. 7 is a block diagram illustrating an example software architecture,which may be used in conjunction with various hardware architecturesdescribed herein; and

FIG. 8 is a block diagram illustrating components of a machine,according to some example embodiments, configured to read instructionsfrom a machine-readable medium (e.g., a machine-readable storage medium)and perform any one or more of the methodologies discussed herein.

It will be noted that throughout the appended drawings, like featuresare identified by like reference numerals.

DETAILED DESCRIPTION

The description that follows describes example systems, methods,techniques, instruction sequences, and computing machine programproducts that comprise illustrative embodiments of the disclosure,individually or in combination. In the following description, for thepurposes of explanation, numerous specific details are set forth inorder to provide an understanding of various embodiments of theinventive subject matter. It will be evident, however, to those skilledin the art, that various embodiments of the inventive subject matter maybe practiced without these specific details.

The term ‘content’ used throughout the description herein should beunderstood to include all forms of media content items, includingimages, videos, audio, text, 3D models (e.g., including textures,materials, meshes, and more), animations, vector graphics, and the like.

The term ‘game’ used throughout the description herein should beunderstood to include video games and applications that execute andpresent video games on a device, and applications that execute andpresent simulations on a device. The term ‘game’ should also beunderstood to include programming code (either source code or executablebinary code) which is used to create and execute the game on a device.

The term ‘environment’ used throughout the description herein should beunderstood to include 2D digital environments (e.g., 2D video gameenvironments, 2D scene environments, 2D simulation environments, 2Dcontent creation environments, and the like), 3D digital environments(e.g., 3D game environments, 3D simulation environments, 3D sceneenvironments, 3D content creation environment, virtual realityenvironments, and the like), and augmented reality environments thatinclude both a digital (e.g., virtual) component and a real-worldcomponent.

The term ‘digital object’, used throughout the description herein isunderstood to include any object of digital nature, digital structure ordigital element within an environment. A digital object can represent(e.g., in a corresponding data structure) almost anything within theenvironment; including 3D models (e.g., characters, weapons, sceneelements (e.g., buildings, trees, cars, treasures, and the like)) with3D model textures, backgrounds (e.g., terrain, sky, and the like),lights, cameras, effects (e.g., sound and visual), animation, and more.The term ‘digital object’ may also be understood to include linkedgroups of individual digital objects. A digital object is associatedwith data that describes properties and behavior for the object.

The terms ‘asset’ and ‘digital asset’, used throughout the descriptionherein are understood to include any data that can be used to describe adigital object or can be used to describe an aspect of a digital project(e.g., including: a game, a film, a software application). For example,an asset can include data for an image, a 3D model (textures, rigging,and the like), a group of 3D models (e.g., an entire scene), an audiosound, a video, animation, a 3D mesh and the like. The data describingan asset may be stored within a file, or may be contained within acollection of files, or may be compressed and stored in one file (e.g.,a compressed file), or may be stored within a memory. The datadescribing an asset can be used to instantiate one or more digitalobjects within a game at runtime (e.g., during execution of the game).

The terms ‘client’ and ‘application client’ used throughout thedescription herein are understood to include a software client orsoftware application that can access data and services on a server,including accessing over a network.

Throughout the description herein, the term ‘mixed reality’ (MR) shouldbe understood to include all combined environments in the spectrumbetween reality and virtual reality (VR) including virtual reality,augmented reality (AR) and augmented virtuality.

A method of rendering an image of an environment is disclosed.Environment data for the environment is accessed. The environment datacorresponds to a frame of a video. A plurality of subframes associatedwith the frame is determined. An angle for each of the plurality ofsubframes is determined. One or more lights corresponding to theenvironment are selected. For each light of the one or more lights, ashadow map is generated. The shadow map corresponds to a subframe of theplurality of subframes based on a frustum view oriented at the angledetermined for the subframe. The image of the environment is rendered.The rendering includes using the generated shadow map for each light ofthe one or more lights.

The present invention includes apparatuses which perform one or moreoperations or one or more combinations of operations described herein,including data processing systems which perform these methods andcomputer readable media which when executed on data processing systemscause the systems to perform these methods, the operations orcombinations of operations including non-routine and unconventionaloperations or combinations of operations.

The systems and methods described herein include one or more componentsor operations that are non-routine or unconventional individually orwhen combined with one or more additional components or operations,because, for example, they provide a number of valuable benefits todigital content creators. For example, the systems and methods describedherein improve a quality of rendered shadows (e.g., reducing jaggededges) in digital content (e.g., movies) without use of shadow filteringtechniques thus improving an efficiency of computation of renderedimages on a computer. The systems and methods described herein may beused to reduce shadow artifacts including jagged shadow edges and lightleaks displayed in moving shadows in digital content such as videos(e.g., videos that use motion blurring techniques), wherein the reducingof the shadow artifacts avoids a use of computationally intensive shadowfiltering techniques.

Turning now to the drawings, systems and methods, including non-routineor unconventional components or operations, or combinations of suchcomponents or operations, for accumulated shadow map rotation inaccordance with embodiments of the invention are illustrated. Inaccordance with an embodiment, FIG. 1 is a diagram of an example shadowmap rotation system 100 and associated devices configured to provideshadow map rotation functionality. In accordance with an embodiment, theshadow map rotation device 104 is a computing device capable ofproviding a digital environment to a user 102. In some embodiments, theshadow map rotation device 104 is a mobile computing device, such as asmartphone, a tablet computer, or a head mounted display (HMD) devicewhile in other embodiments, the shadow map rotation device 104 is acomputing device such as a desktop computer or workstation.

In accordance with an embodiment, the shadow map rotation device 104includes one or more central processing units (CPUs) 106 and graphicsprocessing units (GPUs) 108. The processing device 106 is any type ofprocessor, processor assembly comprising multiple processing elements(not shown), having access to a memory 122 to retrieve instructionsstored thereon, and execute such instructions. Upon execution of suchinstructions, the instructions implement the processing device 106 toperform a series of tasks as described herein in reference to FIG. 2A,FIG. 2B, FIG. 3, FIG. 4A, FIG. 4B, FIG. 5, FIG. 6A and FIG. 6B. Theshadow map rotation device 104 may also include one or more networkingdevices 112 (e.g., wired or wireless network adapters) for communicatingacross a network. The shadow map rotation device 104 may also includeone or more camera devices 114 which may be configured to capturedigital video of the real world near the device 104. The shadow maprotation device 104 may also include one or more sensors 116, such as aglobal positioning system (GPS) receiver (e.g., for determining a GPSlocation of the shadow map rotation device 104), biometric sensors(e.g., for capturing biometric data of the user 102), motion or positionsensors (e.g., for capturing position data of the user 102 or otherobjects), or an audio microphone (e.g., for capturing sound data). Somesensors 116 may be external to the shadow map rotation device 104, andmay be configured to wirelessly communicate with the shadow map rotationdevice 104 (e.g., such as used in the Microsoft Kinect®, Vive Tracker™,MIT's Lidar sensor, or MIT's wireless emotion detector).

The shadow map rotation device 104 also includes one or more inputdevices 118 such as, for example, a mouse, a keyboard, a keypad, a touchscreen, a microphone, a pointing device, a camera, a hand-held device(e.g., hand motion tracking device), and the like, for inputtinginformation in the form of a data signal readable by the processingdevice 106. The shadow map rotation device 104 further includes one ormore display devices 120, such as a touchscreen of a tablet orsmartphone, or lenses or visor of a VR or AR HMD, which may beconfigured to display virtual objects. The display device 120 may bedriven or controlled by one or more GPUs 108. The GPU 108 processesaspects of graphical output that assists in speeding up rendering ofoutput through the display device 120.

The shadow map rotation device 104 also includes a memory 122 configuredto store a shadow map rotation module 126. The memory 122 can be anytype of memory device, such as random access memory, read only orrewritable memory, internal processor caches, and the like. The memory122 also stores an application 124 (e.g., executed by the CPU 106 or GPU108) that communicates with the display device 120 and also with otherhardware such as the input/output device(s) 118 to present a digitalenvironment (e.g., a 3D video game or a 3D content creation environment)on the display device 120. In accordance with an embodiment, theapplication may be a digital content creation application that providestools (e.g., user interfaces via the display device) for creatingdigital content including video games, movies, television shows andmore. In accordance with an embodiment, the application may include acontent creation engine, whereby the content creation engine wouldtypically include one or more modules that provide the following:animation physics for digital objects, collision detection for digitalobjects, rendering, networking, sound, animation, and the like in orderto provide a digital environment for display on the display device 120.In accordance with an embodiment, the application 124 may include arendering module 125 (e.g., within the content creation engine) forrendering parts of a digital environment into an image or series ofimages (e.g., as part of a movie). In accordance with an embodiment, therendering module 125 may include a shadow map rotation module 126 thatperforms operations as described below with respect to FIG. 2A, FIG. 2B,FIG. 3, FIG. 4A, FIG. 4B, FIG. 5, FIG. 6A and FIG. 6B. Although theshadow map rotation module 126 is shown as a part of the renderingmodule 125 and application 124, the shadow map rotation module 126 maybe implemented separately from the application 124 and/or separatelyfrom the rendering module 125 (e.g., as a plugin or as a completelyseparate application).

In some embodiments, the shadow map rotation system 100 and the variousassociated hardware and software components described herein may provideaugmented reality (AR) content instead of, or in addition to, virtualreality (VR) content (e.g., in a mixed reality (MR) environment). Itshould be understood that the systems and methods described herein maybe performed with AR content and VR content, and as such, the scope ofthis disclosure covers both AR and VR applications.

In accordance with an embodiment, and shown in FIG. 2A is a flowchart ofa method 200 for rendering images using a plurality of rotation mapsover a plurality of subframes. The method 200 shown in FIG. 2A may bereferred to herein as the rotated shadow map method. The method 200 maybe used in conjunction with the shadow map rotation system 100 asdescribed with respect to FIG. 1. In various embodiments, some of themethod elements shown may be performed concurrently, in a differentorder than shown, or may be omitted. The method 200 may be used as partof a rendering pipeline (e.g., executing within the rendering module125) to generate one or more rendered images from an environment for aframe (e.g., a rendering of a 3D environment for frames in a movie). Inaccordance with an embodiment, at operation 202 of the method 200, theshadow map rotation module 126 receives a rendering request from therendering pipeline to render a scene from a camera view within theenvironment using the rotated shadow map method 200. The request may beto render an image for a frame. In accordance with an embodiment, therequest includes frame data, which includes one or more of thefollowing: data describing an image to be rendered for the frame, datadescribing shadow map parameters for the frame, and data describingsubframe parameters for the frame. In accordance with an embodiment, thedata describing an image to be rendered may include camera data (e.g.,position, orientation, view frustum, camera parameters, and more inorder to position, orient and frame a frustum view of the camera withinthe environment), and environment data (e.g., 3D model data, texturedata, rendering options, and more). In accordance with an embodiment,the environment data may provide a link to an environment rather thandirectly providing 3D model data and texture data. In accordance with anembodiment, the data describing shadow map parameters includes one ormore of the following data describing a shadow map: shadow mapresolution, shadow map size, shadow map shape (e.g., 2D rectangular, 2Dsquare, and cube), shadow map cascade parameters (e.g., number, size,resolution of each cascade), and more. In accordance with an embodiment,the data describing subframe parameters may include data that describesa number of subframes and rotation angles for the number of subframes.For example, the subframe parameters may include a rotation angle for ashadow map associated with each of the number of subframes, and a totalrotation angle for the shadow maps.

In accordance with an embodiment, the shadow map parameters and thesubframe parameters may be predetermined by an external agent (e.g., arendering engineer, 3D graphic artist, and the like) to provide highquality rendering of shadows using the method 200. For example, thenumber of subframes may be increased to provide a higher quality ofartifact averaging (e.g., during operation 210 described below), or thenumber of subframes may be reduced to reduce an amount of computationduring operation. Similarly, the total rotation angle for the shadowmaps may be increased to provide a larger distribution of artifacts forartifact averaging.

In accordance with an embodiment, at operation 204 of the method 200,the shadow map rotation module 126 determines a rotation angle and aduration for each subframe of the number of subframes. The determinationof the rotation angle for each subframe may be based on the subframeparameter data such that the rotation angle for each subframe isconsistent with the number of subframes and the total rotation angle.Similarly, the determination of the subframe duration may be based onthe subframe parameter data such that the duration of the rotation anglefor each subframe is consistent with the number of subframes and a totalframe duration. For example, based on the subframe parameter dataproviding a total rotation angle of 90 degrees, and the number ofsubframes being 15, then each subframe rotation angle is determined tobe incremented by 6 degrees from a previous subframe rotation angle(e.g., subframe 1 at 6 degrees, subframe 2 at 12 degrees, subframe 3 at18 degrees, subframe 4 at 24 degrees, and the like up to a last subframeat 90 degrees). A first subframe rotation angle may start at zerodegrees or at an offset (e.g., with zero degrees being vertical). Inaccordance with an embodiment, the subframe duration is an amount oftime (e.g., in milliseconds (ms)) between two frames covered by asubframe. The subframe duration may be equal for all subframes, or thesubframe duration may be different for each subframe. Based on thesubframe duration being equal for all subframes, the duration may becalculated by dividing a full frame time (e.g., a time for a singleframe) by the number of subframes. A full frame time can be any durationand is related to a number of full frames per second required by theapplication 124 or by a creator using the application 124 or by adesired output; examples of full frame times include 41.66 ms for 24full frames per second (e.g., typical frame rate for movies), 33.33 msfor full frames per second, 16.66 ms for 60 full frames per second(e.g., typical frame rate for televisions and computer monitors), 11.11ms for 90 frames per second (typical frame rate for mixed reality HMDs),and the like. As an example, based on 60 full frames per second and 15subframe, then each subframe would have a duration of approximately 1.11ms. In accordance with an embodiment, and illustrated in FIG. 3 is avisual representation of a series of three frames (e.g., Frame F1 300,Frame F2 302 and Frame F3 304) wherein a first frame Frame F1 300 isdivided into a plurality of subframes. In the example shown in FIG. 3,Frame F1 300 is divided into 6 subframes (Subframe S1 300A to SubframeS6 300F). In accordance with an embodiment, each subframe of the 6subframes would have an associated duration and angle. The frames shownin FIG. 3 may be part of a video, movie, television program, or othercontent.

In accordance with an embodiment and referring back to FIG. 2A, atoperation 206 of the method 200, the shadow map rotation module 126gathers data describing a set of lights in the environment. Inaccordance with an embodiment, the set of lights includes lights whichare involved in lighting the environment. A list of lights to beincluded in the set of lights may be included within the request fromthe rendering pipeline (e.g., received in operation 202). The datadescribing the set of lights may include a description of propertiesassociated with each light within the set of lights. The description mayinclude the following: data describing a type of light (e.g., spotlight, point light, line light, directional light, plane light, and thelike), data describing a location of the light in the environment, datadescribing an orientation of the light in the environment, datadescribing properties of the light such as color, intensity and the likein order to define an output of light within the environment emanatingfrom the light.

In accordance with an embodiment, at operation 208 of the method 200,for one subframe of the plurality of subframes, the shadow map rotationmodule 126 loops through each light of the set of lights, generates ashadow map for each light based in part on the associated angle for thesubframe, and renders an image for the subframe using all the generatedshadow maps. In accordance with an embodiment, the rendering isperformed by the rendering pipeline using one or more of the following:geometry shaders, vertex shaders, compute shaders, fragment shaders, andmore. The rendering may include post processing effects. In accordancewith an embodiment, as described below and shown in FIG. 2B is aflowchart illustrating additional details included within operation 208.In accordance with an embodiment, as part of operation 208, the shadowmap rotation module 126 may loop through only lights of a predeterminedtype (or a plurality of different types) within the set of lights togenerate a shadow map for each light of the type. For example, theshadow map rotation module 126 may only loop through a set of spotlights within the set of lights (e.g., or only point lights, or onlyplane lights, or the like).

In accordance with an embodiment, at operation 210 of the method 200,the shadow map rotation module 126 adds the rendered image for the onesubframe into a single image using an accumulation buffering technique.The accumulation buffering technique may be any technique that canaccumulate a plurality of images into a single image (e.g., using abuffer). In accordance with an embodiment, operation 210 is performedfor a plurality of rendered images (e.g., using one rendered image foreach subframe determined in operation 204) creating an accumulated oraveraged single image for the plurality of subframes. In accordance withan embodiment, the accumulating of rendered images in operation 210 alsoaccumulates (e.g., and averages) noise and artifacts within shadowsgenerated during the rendering of each subframe (e.g., with eachsubframe having a shadow map at a different angle). The noise andartifacts within rendered shadows of a rendered image are based in parton shadow map angles for each light source used in the rendering.Accordingly, each rendering performed for the various rotated shadowmaps in operation 208 may add unique visual noise and artifacts withinshadows of the accumulated image. The accumulating of the noise andartifacts (e.g., within shadows) helps to smooth out rendered shadows inthe accumulated single image.

In accordance with an embodiment, at operation 212 of the method 200,the operations 208 and 210 are looped for each subframe such that eachsubframe goes through operation 208 and 210. In accordance with anembodiment the looping is done progressively from a first subframe(e.g., Subframe S1 300A) to a last subframe (e.g., Subframe S6 300F)with increasing associated angles. In accordance with an embodiment, aspart of the looping, at operation 212, based on the rendering of thesubframes not being complete, a next subframe is selected and operation208 and 210 are repeated on the next subframe (e.g., with an associatednext angle) leading to an updated single image in operation 210.

In accordance with an embodiment, at operation 214 of the method 200,based on a completion of processing of the plurality of subframesdetermined in operation 204, the shadow map rotation module 126 exportsa final single image for the frame, wherein the final single image isthe accumulated rendered image generated in operation 210. The finalsingle image may be exported back to the rendering pipeline for furtherprocessing (e.g., image post processing), including using the finalsingle image as a frame (e.g., Frame 1 300) in a video. Optionally, ifthere is a next frame to process using the method 200, the shadow maprotation module 126 may loop back to operation 202 and work on the nextframe. In accordance with an embodiment, based on completion of theaccumulated single image (e.g., including the exporting of the image),the shadow map rotation module 126 may clear a buffer associated withthe accumulation of the image (e.g., a buffer used in operation 210 maybe cleared prior to working on the next frame).

In accordance with an embodiment, and shown in FIG. 2B, are additionaldetails describing operation 208. In accordance with an embodiment, atoperation 208A, the shadow map rotation module 126 rotates a frustumview associated with a light of the set of lights, wherein the rotationbrings the frustum view to the associated angle of the subframe. Inaccordance with an embodiment, the rotation of the frustum view does notrotate the light itself (e.g., the distribution of light within theenvironment is unchanged), but rather the rotation of the frustum viewrotates a view into the environment from a point associated with thelight, wherein the frustum view is used to generate a shadow map. Inoperation 208A, the shadow map rotation module 126 may rotate thefrustum view for each light of the set of lights to the associated angleof the subframe.

In accordance with an embodiment, at operation 208B within operation208, the shadow map rotation module 126 generates a rotated shadow mapfor each light of the set of lights. In accordance with an embodiment,the rotated shadow map for a light is generated based on shadow mapparameters for the light, properties for the light, and the rotationangle associated with the subframe (e.g., using the rotated frustumview). The rotated frustum view is used to generate the rotated shadowmap (e.g., a texture) from a point of view of the light looking into theenvironment via the rotated frustum view. In accordance with anembodiment, the rotated shadow map may have a similar data structure asa texture, and the rotated shadow map for the light may be generated ina similar way to how a camera generates a depth texture. For example,consider a camera positioned in a same location as the light, andlooking through the rotated frustum, areas of the environment thatcannot be seen are the same areas of the environment that rays from thelight cannot reach; therefore, they are in shadow, and are recorded assuch in the shadow map. In accordance with an embodiment, the shadow maprotation module 126 may populate the shadow map with information about adistance travelled by rays from the light before hitting a surface. Eachrotation of the frustum view for a light will generate a differentshadow map because of a grid-like nature of a shadow map texture (e.g.,even while the frustum view is pointed towards a same part of theenvironment from a same position of the light). Each point in the gridis referred to as a texel (e.g., similar to a pixel of an image). Forexample, the shadow map for a light may be generated by capturing shadowdata in a grid format (e.g., a grid of texels) looking through therotated frustum, wherein the grid is rotated along with the frustum(e.g., as described with respect to FIG. 4A and FIG. 4B). Data withineach texel is an average of shadow data from a part of the environmentseen through one part of the grid via the rotated frustum. A lowerresolution shadow mask has larger grid elements (e.g., larger texels)and includes averaging from a larger part of the environment and mayproduce lower quality shadows when used in rendering (e.g., withinoperation 208C). A mismatch between texel size in a shadow map andscreen pixel size in a rendered image using the shadow map can lead toartifacts in the rendered image (e.g., when the texel is larger than thepixel). The rotation of the frustum to generate rotated shadow mapsperformed within operation 208B is performed in part to average theartifacts and produce higher quality rendered images (e.g., regardlessof texel size).

In accordance with an embodiment, at operation 208C, the shadow maprotation module 126 provides the generated rotated shadow maps (e.g.,from operation 208B) for the subframe to the render pipeline forrendering an image from the camera view (e.g., the camera view describedin the rendering request received in operation 202). In accordance withan embodiment, the render pipeline uses the provided rotated shadow mapsfor the subframe to render the image for the subframe. The rendering mayinclude rendering geometry, post processing the image and more. Inaccordance with an embodiment, shadows in the rendered image may begenerated using the rotated shadow maps within a fragment shaderoperation within the rendering operation. In accordance with anembodiment, operation 208C allows the render pipeline to control aspectsof rendering the image with the exception of the generation of shadowmaps. In accordance with an embodiment, operation 208C may includefiltering of the shadow maps and filtering of shadows generated in arendered image during the rendering process. However, filtering (e.g.,percentage-closer filtering PCF) may be performed at a cost of losingprecision, sharpness and introducing light leak in the rendered image.Various operations in the method 200, including operation 208C may becompatible with filtering, but can be performed without filtering. Inaccordance with an embodiment, filtering may be a configurable option(e.g., set by a programmer or administrator).

In accordance with an embodiment, during a subframe loop for a singleframe (e.g., including operation 208, operation 210 and operation 212),the environment, the camera view and the lights do not change. Forexample, based on the subframes in a subframe loop being linked to thesingle frame, the environment data, light data and camera view data doesnot change (e.g., the data is constant for all the subframes of thesubframe loop).

In accordance with an embodiment, FIG. 4A is an illustration of afrustum view of an instance of a scene in an environment 400 from apoint of view of a light. In accordance with an embodiment, FIG. 4Arepresents a first subframe for a frame. In accordance with anembodiment, the instance of the scene includes a character 404 at thetop of a set of stairs. The frustum view is denoted by a grid 402A andis meant as an illustration of a frustum view that is used to generate ashadow map of the environment 400 for the light (e.g., generated asdescribed in operation 208B), wherein the shadow map is associated withthe first subframe. The grid 402 represents a window view into theenvironment 400 from the point of view of the light. A boundary 406 inFIG. 4A is shown as an example boundary for direct light rays emanatingfrom the light and interacting with the environment 400 and creatingshadows. The boundary 406 shown in FIG. 4A is shaped as a circle basedon the light being a spotlight; however, the boundary 406 may take onany arbitrary shape based on a type of the light. In accordance with anembodiment, a minimum size of the frustum view may be determined toensure that the entire boundary 406 is included in the frustum view. InFIG. 4A, the grid 402A is horizontal and may represent a starting pointfor a first subframe in the subframe loop wherein the frustum ishorizontal (e.g., as described in operation 208A). The squares in thegrid 402A represent texels in the shadow map and are shown for ease ofexplanation. Furthermore, the size of the squares in the grid 402A maybe exaggerated for ease of explanation. In accordance with anembodiment, and as shown in FIG. 4A, the boundary 406 and the grid 402may be of different shape. Improved performance of the method 200 may beachieved when the grid 402 fully includes the boundary 406 for allsubframe rotation angles (e.g., during operations 208, 210 and 212).

In accordance with an embodiment, FIG. 4B is an illustration of a secondrotated frustum view of the same instance of the scene from FIG. 4A fromthe point of view of the light. In accordance with an embodiment, FIG.4B represents a second subframe for the frame shown in FIG. 4A. Thesecond rotated frustum view is denoted by a rotated grid 402B and ismeant as an illustration of the second rotated frustum view that is usedto generate (e.g., generated as described in operation 208B) a rotatedshadow map of the environment 400 for the light), wherein the rotatedshadow map is associated with the second subframe. In FIG. 4B the gridis rotated and may represent a subframe within the subframe loop whereinthe frustum has been rotated by approximately 45 degrees from horizontal(e.g., as described in operation 208A). The squares (e.g., texels) inthe rotated grid 402B capture shadows in the scene 400 (e.g., capture ina shadow map) from a different orientation which may lead to ageneration of different artifacts in a rendered image of the scene whencompared to artifacts generated from a shadow map created with the grid402A from FIG. 4A. The boundary 406 shown in FIG. 4B is unchanged fromFIG. 4A since the light has not moved (e.g., including a lack ofrotation) between the first subframe and the second subframe (e.g.,based on the environment (including lights) remaining unchanged for allsubframes of a frame).

In accordance with an embodiment, the grid 402A in FIG. 4A and therotated grid 402B in FIG. 4B represent a uniform two-dimensional (2D)shadow map wherein the resolution of the shadow map is uniform. Auniform 2D shadow map may be used to represent light from a spotlight.Other types of shadow maps may be used to represent other types oflights (e.g., directional lights, point lights, planar lights, and thelike) in the operations described with respect to the method 200 shownin FIG. 2A and FIG. 2B. For example, cascaded shadow maps which have aplurality of sections, wherein each section has a different resolution,may also be used in the operations of the method 200. A cascaded shadowmap may be used to represent light for directional light (e.g., lightwhich illuminates an environment uniformly from a single direction andwhich appears to come from infinitely far away; e.g., such as sunlight).

In accordance with an embodiment, taking into consideration that aspotlight (e.g., which generates a cone of light within the environment)only generates shadows within a circular area inside of a square shadowmap, the subframe rotation of operation 208 and the accumulation ofoperation 210 can be applied with a grid (as shown in FIG. 4A and FIG.4B) without losing shadow coverage. In accordance with an embodiment,and based on a type of light, operation 208 may be performed using amask to block out shadows outside of a circle similar to the circle inFIG. 4A and FIG. 4B. In accordance with an embodiment, the mask may becircular and be similar to the boundary 406 in FIG. 4A and FIG. 4B. Forexample, a mask could be used to render shadows within a boundary of themask using the method 200 shown in FIG. 2A and FIG. 2B, while shadowsoutside of the mask boundary (e.g., outside of the boundary 406) may berendered using alternative methods. As an example, and in accordancewith an embodiment, a mask may be used with a directional type of lightsince the light type would fill any shape or size of frustum leading topossible edge artifacts for non-circular frustums, which would bemitigated with a mask. In accordance with an embodiment, an increase ina number of subframes and subframe angles may reduce an amount ofartifacts due to a non-circular frustum.

In accordance with an embodiment, though shown in FIG. 4A and FIG. 4Bwith a rectangular frustum view, the method 200 may use any shapedfrustum view in operation 208. For example, a rotationally symmetricfrustum view may provide shadows with fewer artifacts.

In accordance with an embodiment, and shown in FIG. 5, is an unfoldedcube shaped shadow map 500 which may be used in the operations of themethod 200 (e.g., during operation 208). A cube shaped shadow map may beconstructed as a shadow map for a point light source based on light fromthe point source traveling in all directions within an environment, andwherein shadow maps on the surface of the cube are determined with thepoint light at a center of the cube. In accordance with an embodiment,the cube shadow map may be determined using six frustums pointing in thesix directions of the cube faces and with each frustum originating froma position of the point light (e.g., one frustum for each cube face). Inaccordance with an embodiment, the cube shadow map 500 for a light in anenvironment may include six 2D shadow maps representing six faces of thecube (e.g., six 2D shadow map textures), the six faces representing sixdirections of light travel wherein each of the six faces is similar to a2D shadow map for a direction associated with a face, and contains agrid of texels (grid not shown in FIG. 5). For example, there may be a‘Bottom’ 2D shadow map 500F within the cubemap 500 representing a shadowmap of the environment in a downward direction from the light (e.g.,determined using a downward facing frustum). Similarly, there may be a‘Top’ 2D shadow map 500E within the cubemap 500 representing a shadowmap of the environment in an upward direction from the light (e.g.,determined using an upward facing frustum). Similarly, there may be a‘Left’ 2D shadow map 500A, a ‘Front’ 2D shadow map 500B, a ‘Right’ 2Dshadow map 500C and a ‘Back’ 2D shadow map 500D within the cubemap 500representing 4 horizontal directions from the light (e.g., determinedusing 4 horizontally directed frustums). In accordance with anembodiment, FIG. 5 also shows a possible rotated cubemap 502 wherein theentire cubemap 500 has been rotated (e.g., as part of operation 208) asa unit about the point light (e.g., rather than each individual cubeface being rotated).

In accordance with an embodiment, FIG. 6A is an illustration of arendered image 600 for a single subframe within the method 200 (e.g., arendered image from operation 208C). The rendered image 600 mayrepresent a close-up shot of the character 404 shown in FIGS. 4A and 4B.The rendered image shows a plurality of shadows with jagged edges (602A,604A, 606A and 608A) on the character 404. The jagged edges on theplurality of shadows (602A, 604A, 606A and 608A) are artifacts ofrendering using a shadow map generated in operation 208B for a singlesubframe (e.g., wherein the shadow map may have been generated from afrustum view at a first angle). A second rendered image (not shown inFIG. 6A) of the same frame but for a second subframe (e.g., wherein ashadow map for the second subframe may have been generated from afrustum view rotated to a second angle) may have slightly differentartifacts with different jagged edges.

In accordance with an embodiment, FIG. 6B is an illustration of anaccumulated (e.g., averaged) rendered image 610 for a plurality ofsubframes within the method 200. For example, the accumulated renderedimage 610 may be generated as part of operation 210 of the method 200(e.g., and used in operation 214). The accumulated image 610 is anaveraging of a plurality of images rendered for each subframe (e.g.,image 600). The accumulated image 610 illustrates an averaging of shadowartifacts from each rendered subframe image (e.g., image 600). Aplurality of smooth shadows (602B, 604B, 606B and 608B) represent anaveraging of the jagged shadows (602A, 604A, 606A and 608A) within eachsubframe respectively.

While illustrated in the block diagrams as groups of discrete componentscommunicating with each other via distinct data signal connections, itwill be understood by those skilled in the art that the variousembodiments may be provided by a combination of hardware and softwarecomponents, with some components being implemented by a given functionor operation of a hardware or software system, and many of the datapaths illustrated being implemented by data communication within acomputer application or operating system. The structure illustrated isthus provided for efficiency of teaching the present variousembodiments.

It should be noted that the present disclosure can be carried out as amethod, can be embodied in a system, a computer readable medium or anelectrical or electro-magnetic signal. The embodiments described aboveand illustrated in the accompanying drawings are intended to beexemplary only. It will be evident to those skilled in the art thatmodifications may be made without departing from this disclosure. Suchmodifications are considered as possible variants and lie within thescope of the disclosure.

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied on a machine-readable medium or ina transmission signal) or hardware modules. A “hardware module” is atangible unit capable of performing certain operations and may beconfigured or arranged in a certain physical manner. In various exampleembodiments, one or more computer systems (e.g., a standalone computersystem, a client computer system, or a server computer system) or one ormore hardware modules of a computer system (e.g., a processor or a groupof processors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In some embodiments, a hardware module may be implemented mechanically,electronically, or with any suitable combination thereof. For example, ahardware module may include dedicated circuitry or logic that ispermanently configured to perform certain operations. For example, ahardware module may be a special-purpose processor, such as afield-programmable gate array (FPGA) or an Application SpecificIntegrated Circuit (ASIC). A hardware module may also includeprogrammable logic or circuitry that is temporarily configured bysoftware to perform certain operations. For example, a hardware modulemay include software encompassed within a general-purpose processor orother programmable processor. Such software may at least temporarilytransform the general-purpose processor into a special-purposeprocessor. It will be appreciated that the decision to implement ahardware module mechanically, in dedicated and permanently configuredcircuitry, or in temporarily configured circuitry (e.g., configured bysoftware) may be driven by cost and time considerations.

Accordingly, the phrase “hardware module” should be understood toencompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired), or temporarilyconfigured (e.g., programmed) to operate in a certain manner or toperform certain operations described herein. As used herein,“hardware-implemented module” refers to a hardware module. Consideringembodiments in which hardware modules are temporarily configured (e.g.,programmed), each of the hardware modules need not be configured orinstantiated at any one instance in time. For example, where a hardwaremodule comprises a general-purpose processor configured by software tobecome a special-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware modules) at different times. Software mayaccordingly configure a particular processor or processors, for example,to constitute a particular hardware module at one instance of time andto constitute a different hardware module at a different instance oftime.

Hardware modules can provide information to, and receive informationfrom, other hardware modules. Accordingly, the described hardwaremodules may be regarded as being communicatively coupled. Where multiplehardware modules exist contemporaneously, communications may be achievedthrough signal transmission (e.g., over appropriate circuits and buses)between or among two or more of the hardware modules. In embodiments inwhich multiple hardware modules are configured or instantiated atdifferent times, communications between such hardware modules may beachieved, for example, through the storage and retrieval of informationin memory structures to which the multiple hardware modules have access.For example, one hardware module may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware module may then, at a latertime, access the memory device to retrieve and process the storedoutput. Hardware modules may also initiate communications with input oroutput devices, and can operate on a resource (e.g., a collection ofinformation).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions describedherein. As used herein, “processor-implemented module” refers to ahardware module implemented using one or more processors.

Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors orprocessor-implemented modules. Moreover, the one or more processors mayalso operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an application programinterface (API)).

The performance of certain of the operations may be distributed amongthe processors, not only residing within a single machine, but deployedacross a number of machines. In some example embodiments, the processorsor processor-implemented modules may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented modules may be distributed across a number ofgeographic locations.

FIG. 7 is a block diagram 700 illustrating an example softwarearchitecture 702, which may be used in conjunction with various hardwarearchitectures herein described to provide a gaming engine 701 and/orcomponents of the shadow map rotation system 100. FIG. 7 is anon-limiting example of a software architecture and it will beappreciated that many other architectures may be implemented tofacilitate the functionality described herein. The software architecture702 may execute on hardware such as a machine 800 of FIG. 8 thatincludes, among other things, processors 810, memory 830, andinput/output (I/O) components 850. A representative hardware layer 704is illustrated and can represent, for example, the machine 800 of FIG.8. The representative hardware layer 704 includes a processing unit 706having associated executable instructions 708. The executableinstructions 708 represent the executable instructions of the softwarearchitecture 702, including implementation of the methods, modules andso forth described herein. The hardware layer 704 also includesmemory/storage 710, which also includes the executable instructions 708.The hardware layer 704 may also comprise other hardware 712.

In the example architecture of FIG. 7, the software architecture 702 maybe conceptualized as a stack of layers where each layer providesparticular functionality. For example, the software architecture 702 mayinclude layers such as an operating system 714, libraries 716,frameworks or middleware 718, applications 720 and a presentation layer744. Operationally, the applications 720 and/or other components withinthe layers may invoke application programming interface (API) calls 724through the software stack and receive a response as messages 726. Thelayers illustrated are representative in nature and not all softwarearchitectures have all layers. For example, some mobile or specialpurpose operating systems may not provide the frameworks/middleware 718,while others may provide such a layer. Other software architectures mayinclude additional or different layers.

The operating system 714 may manage hardware resources and providecommon services. The operating system 714 may include, for example, akernel 728, services 730, and drivers 732. The kernel 728 may act as anabstraction layer between the hardware and the other software layers.For example, the kernel 728 may be responsible for memory management,processor management (e.g., scheduling), component management,networking, security settings, and so on. The services 730 may provideother common services for the other software layers. The drivers 732 maybe responsible for controlling or interfacing with the underlyinghardware. For instance, the drivers 732 may include display drivers,camera drivers, Bluetooth® drivers, flash memory drivers, serialcommunication drivers (e.g., Universal Serial Bus (USB) drivers), Wi-Fi®drivers, audio drivers, power management drivers, and so forth dependingon the hardware configuration.

The libraries 716 may provide a common infrastructure that may be usedby the applications 720 and/or other components and/or layers. Thelibraries 716 typically provide functionality that allows other softwaremodules to perform tasks in an easier fashion than to interface directlywith the underlying operating system 714 functionality (e.g., kernel728, services 730 and/or drivers 732). The libraries 816 may includesystem libraries 734 (e.g., C standard library) that may providefunctions such as memory allocation functions, string manipulationfunctions, mathematic functions, and the like. In addition, thelibraries 716 may include API libraries 736 such as media libraries(e.g., libraries to support presentation and manipulation of variousmedia format such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG), graphicslibraries (e.g., an OpenGL framework that may be used to render 2D and3D graphic content on a display), database libraries (e.g., SQLite thatmay provide various relational database functions), web libraries (e.g.,WebKit that may provide web browsing functionality), and the like. Thelibraries 716 may also include a wide variety of other libraries 738 toprovide many other APIs to the applications 720 and other softwarecomponents/modules.

The frameworks 718 (also sometimes referred to as middleware) provide ahigher-level common infrastructure that may be used by the applications720 and/or other software components/modules. For example, theframeworks/middleware 718 may provide various graphic user interface(GUI) functions, high-level resource management, high-level locationservices, and so forth. The frameworks/middleware 718 may provide abroad spectrum of other APIs that may be utilized by the applications720 and/or other software components/modules, some of which may bespecific to a particular operating system or platform.

The applications 720 include built-in applications 740 and/orthird-party applications 742. Examples of representative built-inapplications 740 may include, but are not limited to, a contactsapplication, a browser application, a book reader application, alocation application, a media application, a messaging application,and/or a game application. Third-party applications 742 may include anyan application developed using the Android™ or iOS™ software developmentkit (SDK) by an entity other than the vendor of the particular platform,and may be mobile software running on a mobile operating system such asiOS™, Android™, Windows® Phone, or other mobile operating systems. Thethird-party applications 742 may invoke the API calls 724 provided bythe mobile operating system such as operating system 714 to facilitatefunctionality described herein.

The applications 720 may use built-in operating system functions (e.g.,kernel 728, services 730 and/or drivers 732), libraries 716, orframeworks/middleware 718 to create user interfaces to interact withusers of the system. Alternatively, or additionally, in some systems,interactions with a user may occur through a presentation layer, such asthe presentation layer 744. In these systems, the application/module“logic” can be separated from the aspects of the application/module thatinteract with a user.

Some software architectures use virtual machines. In the example of FIG.7, this is illustrated by a virtual machine 748. The virtual machine 748creates a software environment where applications/modules can execute asif they were executing on a hardware machine (such as the machine 800 ofFIG. 8, for example). The virtual machine 748 is hosted by a hostoperating system (e.g., operating system 714) and typically, althoughnot always, has a virtual machine monitor 746, which manages theoperation of the virtual machine 748 as well as the interface with thehost operating system (i.e., operating system 714). A softwarearchitecture executes within the virtual machine 748 such as anoperating system (OS) 750, libraries 752, frameworks 754, applications756, and/or a presentation layer 758. These layers of softwarearchitecture executing within the virtual machine 748 can be the same ascorresponding layers previously described or may be different.

FIG. 8 is a block diagram illustrating components of a machine 800,according to some example embodiments, configured to read instructionsfrom a machine-readable medium (e.g., a machine-readable storage medium)and perform any one or more of the methodologies discussed herein. Insome embodiments, the machine 800 is similar to the shadow map rotationdevice 104. Specifically, FIG. 8 shows a diagrammatic representation ofthe machine 800 in the example form of a computer system, within whichinstructions 816 (e.g., software, a program, an application, an applet,an app, or other executable code) for causing the machine 800 to performany one or more of the methodologies discussed herein may be executed.As such, the instructions 816 may be used to implement modules orcomponents described herein. The instructions transform the general,non-programmed machine into a particular machine programmed to carry outthe described and illustrated functions in the manner described. Inalternative embodiments, the machine 800 operates as a standalone deviceor may be coupled (e.g., networked) to other machines. In a networkeddeployment, the machine 800 may operate in the capacity of a servermachine or a client machine in a server-client network environment, oras a peer machine in a peer-to-peer (or distributed) networkenvironment. The machine 800 may comprise, but not be limited to, aserver computer, a client computer, a personal computer (PC), a tabletcomputer, a laptop computer, a netbook, a set-top box (STB), a personaldigital assistant (PDA), an entertainment media system, a cellulartelephone, a smart phone, a mobile device, a wearable device (e.g., asmart watch), a smart home device (e.g., a smart appliance), other smartdevices, a web appliance, a network router, a network switch, a networkbridge, or any machine capable of executing the instructions 816,sequentially or otherwise, that specify actions to be taken by themachine 800. Further, while only a single machine 800 is illustrated,the term “machine” shall also be taken to include a collection ofmachines that individually or jointly execute the instructions 816 toperform any one or more of the methodologies discussed herein.

The machine 800 may include processors 810, memory 830, and input/output(I/O) components 850, which may be configured to communicate with eachother such as via a bus 802. In an example embodiment, the processors810 (e.g., a Central Processing Unit (CPU), a Reduced Instruction SetComputing (RISC) processor, a Complex Instruction Set Computing (CISC)processor, a Graphics Processing Unit (GPU), a Digital Signal Processor(DSP), an Application Specific Integrated Circuit (ASIC), aRadio-Frequency Integrated Circuit (RFIC), another processor, or anysuitable combination thereof) may include, for example, a processor 812and a processor 814 that may execute the instructions 816. The term“processor” is intended to include multi-core processor that maycomprise two or more independent processors (sometimes referred to as“cores”) that may execute instructions contemporaneously. Although FIG.8 shows multiple processors, the machine 800 may include a singleprocessor with a single core, a single processor with multiple cores(e.g., a multi-core processor), multiple processors with a single core,multiple processors with multiples cores, or any combination thereof.

The memory/storage 830 may include a memory, such as a main memory 832,a static memory 834, or other memory, and a storage unit 836, bothaccessible to the processors 810 such as via the bus 802. The storageunit 836 and memory 832, 834 store the instructions 816 embodying anyone or more of the methodologies or functions described herein. Theinstructions 816 may also reside, completely or partially, within thememory 832, 834, within the storage unit 836, within at least one of theprocessors 810 (e.g., within the processor's cache memory), or anysuitable combination thereof, during execution thereof by the machine800. Accordingly, the memory 832, 834, the storage unit 836, and thememory of processors 810 are examples of machine-readable media 838.

As used herein, “machine-readable medium” means a device able to storeinstructions and data temporarily or permanently and may include, but isnot limited to, random-access memory (RAM), read-only memory (ROM),buffer memory, flash memory, optical media, magnetic media, cachememory, other types of storage (e.g., Erasable Programmable Read-OnlyMemory (EEPROM)) and/or any suitable combination thereof. The term“machine-readable medium” should be taken to include a single medium ormultiple media (e.g., a centralized or distributed database, orassociated caches and servers) able to store the instructions 816. Theterm “machine-readable medium” shall also be taken to include anymedium, or combination of multiple media, that is capable of storinginstructions (e.g., instructions 816) for execution by a machine (e.g.,machine 800), such that the instructions, when executed by one or moreprocessors of the machine 800 (e.g., processors 810), cause the machine800 to perform any one or more of the methodologies or operations,including non-routine or unconventional methodologies or operations, ornon-routine or unconventional combinations of methodologies oroperations, described herein. Accordingly, a “machine-readable medium”refers to a single storage apparatus or device, as well as “cloud-based”storage systems or storage networks that include multiple storageapparatus or devices. The term “machine-readable medium” excludessignals per se.

The input/output (I/O) components 850 may include a wide variety ofcomponents to receive input, provide output, produce output, transmitinformation, exchange information, capture measurements, and so on. Thespecific input/output (I/O) components 850 that are included in aparticular machine will depend on the type of machine. For example,portable machines such as mobile phones will likely include a touchinput device or other such input mechanisms, while a headless servermachine will likely not include such a touch input device. It will beappreciated that the input/output (I/O) components 850 may include manyother components that are not shown in FIG. 8. The input/output (I/O)components 850 are grouped according to functionality merely forsimplifying the following discussion and the grouping is in no waylimiting. In various example embodiments, the input/output (I/O)components 850 may include output components 852 and input components854. The output components 852 may include visual components (e.g., adisplay such as a plasma display panel (PDP), a light emitting diode(LED) display, a liquid crystal display (LCD), a projector, or a cathoderay tube (CRT)), acoustic components (e.g., speakers), haptic components(e.g., a vibratory motor, resistance mechanisms), other signalgenerators, and so forth. The input components 854 may includealphanumeric input components (e.g., a keyboard, a touch screenconfigured to receive alphanumeric input, a photo-optical keyboard, orother alphanumeric input components), point based input components(e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, oranother pointing instrument), tactile input components (e.g., a physicalbutton, a touch screen that provides location and/or force of touches ortouch gestures, or other tactile input components), audio inputcomponents (e.g., a microphone), and the like.

In further example embodiments, the input/output (I/O) components 850may include biometric components 856, motion components 858,environmental components 860, or position components 862, among a widearray of other components. For example, the biometric components 856 mayinclude components to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram basedidentification), and the like. The motion components 858 may includeacceleration sensor components (e.g., accelerometer), gravitation sensorcomponents, rotation sensor components (e.g., gyroscope), and so forth.The environmental components 860 may include, for example, illuminationsensor components (e.g., photometer), temperature sensor components(e.g., one or more thermometers that detect ambient temperature),humidity sensor components, pressure sensor components (e.g.,barometer), acoustic sensor components (e.g., one or more microphonesthat detect background noise), proximity sensor components (e.g.,infrared sensors that detect nearby objects), gas sensors (e.g., gasdetection sensors to detection concentrations of hazardous gases forsafety or to measure pollutants in the atmosphere), or other componentsthat may provide indications, measurements, or signals corresponding toa surrounding physical environment. The position components 862 mayinclude location sensor components (e.g., a Global Position System (GPS)receiver component), altitude sensor components (e.g., altimeters orbarometers that detect air pressure from which altitude may be derived),orientation sensor components (e.g., magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The input/output (I/O) components 850 may include communicationcomponents 864 operable to couple the machine 800 to a network 880 ordevices 870 via a coupling 882 and a coupling 872 respectively. Forexample, the communication components 864 may include a networkinterface component or other suitable device to interface with thenetwork 880. In further examples, the communication components 864 mayinclude wired communication components, wireless communicationcomponents, cellular communication components, Near Field Communication(NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy),Wi-Fi® components, and other communication components to providecommunication via other modalities. The devices 870 may be anothermachine or any of a wide variety of peripheral devices (e.g., aperipheral device coupled via a Universal Serial Bus (USB)).

Moreover, the communication components 864 may detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 864 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2Dbar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information may be derived via the communication components862, such as, location via Internet Protocol (IP) geo-location, locationvia Wi-Fi® signal triangulation, location via detecting a NFC beaconsignal that may indicate a particular location, and so forth.

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

The embodiments illustrated herein are described in sufficient detail toenable those skilled in the art to practice the teachings disclosed.Other embodiments may be used and derived therefrom, such thatstructural and logical substitutions and changes may be made withoutdeparting from the scope of this disclosure. The Detailed Description,therefore, is not to be taken in a limiting sense, and the scope ofvarious embodiments is defined only by the appended claims, along withthe full range of equivalents to which such claims are entitled.

As used herein, the term “or” may be construed in either an inclusive orexclusive sense. Moreover, plural instances may be provided forresources, operations, or structures described herein as a singleinstance. Additionally, boundaries between various resources,operations, modules, engines, and data stores are somewhat arbitrary,and particular operations are illustrated in a context of specificillustrative configurations. Other allocations of functionality areenvisioned and may fall within a scope of various embodiments of thepresent disclosure. In general, structures and functionality presentedas separate resources in the example configurations may be implementedas a combined structure or resource. Similarly, structures andfunctionality presented as a single resource may be implemented asseparate resources. These and other variations, modifications,additions, and improvements fall within the scope of embodiments of thepresent disclosure as represented by the appended claims. Thespecification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense.

1. A system comprising: one or more computer processors; one or more computer memories; a set of instructions stored in the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations comprising: accessing environment data for an environment corresponding to a frame of a video; determining a plurality of subframes associated with the frame; determining an angle for each of the plurality of subframes; selecting one or more lights corresponding to the environment; generating, for each light of the one or more lights, a shadow map, the shadow map corresponding to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe; and rendering an image of the environment, the rendering including using the generated shadow map for each light of the one or more lights.
 2. The system of claim 1, the operations further comprising: generating, for each light of the one or more lights, an additional shadow map, the additional shadow map corresponding to an additional subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the additional subframe; and wherein the rendering includes using the additional generated shadow map for each light of the one or more lights.
 3. The system of claim 2, the operations further comprising: creating an averaged single image for the subframe and the additional subframe; and using the averaged single frame as the image.
 4. The system of claim 1, wherein the selecting of the one or more lights is based on a type of the light.
 5. The system of claim 1, wherein the type of the light is one of a spot light, point light, line light, directional light, or plane light.
 6. The system of claim 1, wherein the frustum view dimensions are determined in order to include a predetermined percentage of light output interacting with the environment, the light output emanating from the light of the one or more lights, and wherein the determination may additionally be based one or more of the following: a type of the light of the one or more lights or data describing the light output.
 7. The system of claim 3, wherein the creating of the averaged single image includes adding a rendered image and an additional rendered image into an accumulation buffer, the rendered image corresponding to the subframe and the additional rendered image corresponding to the additional subframe.
 8. The system of claim 1, wherein the rendering is from a camera frustum view.
 9. The system of claim 1, wherein the rendering is performed by a module that is separate from a rendering pipeline, the module configured to supply the environment data to the rendering pipeline.
 10. A non-transitory computer-readable storage medium storing a set of instructions that, when executed by one or more computer processors, causes the one or more computer processors to perform operations, the operations comprising: accessing environment data for an environment corresponding to a frame of a video; determining a plurality of subframes associated with the frame; determining an angle for each of the plurality of subframes; selecting one or more lights corresponding to the environment; generating, for each light of the one or more lights, a shadow map, the shadow map corresponding to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe; and rendering an image of the environment, the rendering including using the generated shadow map for each light of the one or more lights.
 11. The non-transitory computer-readable storage medium of claim 10, the operations further comprising: generating, for each light of the one or more lights, an additional shadow map, the additional shadow map corresponding to an additional subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the additional subframe; and wherein the rendering includes using the additional generated shadow map for each light of the one or more lights.
 12. The non-transitory computer-readable storage medium of claim 11, the operations further comprising: creating an averaged single image for the subframe and the additional subframe; and using the averaged single frame as the image.
 13. The non-transitory computer-readable storage medium of claim 10, wherein the selecting of the one or more lights is based on a type of the light.
 14. The non-transitory computer-readable storage medium of claim 1, wherein the type of the light is one of a spot light, point light, line light, directional light, or plane light.
 15. The non-transitory computer-readable storage medium of claim 10, wherein the frustum view dimensions are determined in order to include a predetermined percentage of light output interacting with the environment, the light output emanating from the light of the one or more lights, and wherein the determination may additionally be based one or more of the following: a type of the light of the one or more lights or data describing the light output.
 16. The non-transitory computer-readable storage medium of claim 12, wherein the creating of the averaged single image includes adding a rendered image and an additional rendered image into an accumulation buffer, the rendered image corresponding to the subframe and the additional rendered image corresponding to the additional subframe.
 17. The non-transitory computer-readable storage medium of claim 10, wherein the rendering is from a camera frustum view.
 18. The non-transitory computer-readable storage medium of claim 10, wherein the rendering is performed by a module that is separate from a rendering pipeline, the module configured to supply the environment data to the rendering pipeline.
 19. A method comprising: accessing environment data for an environment corresponding to a frame of a video; determining a plurality of subframes associated with the frame; determining an angle for each of the plurality of subframes; selecting one or more lights corresponding to the environment; generating, for each light of the one or more lights, a shadow map, the shadow map corresponding to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe; and rendering an image of the environment, the rendering including using the generated shadow map for each light of the one or more lights.
 20. The method of claim 19, further comprising: generating, for each light of the one or more lights, an additional shadow map, the additional shadow map corresponding to an additional subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the additional subframe; and wherein the rendering includes using the additional generated shadow map for each light of the one or more lights. 